{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# 基于图神经网络的小批量图分类任务\n",
    "\n",
    "参考：\n",
    "\n",
    "包括本课程在内的多数pyg教程都来自官网的案例：\n",
    "\n",
    "https://pytorch-geometric.readthedocs.io/en/latest/get_started/colabs.html\n",
    "\n",
    "下面是中文翻译：\n",
    "\n",
    "https://zhuanlan.zhihu.com/p/477339992\n",
    "\n",
    "\n",
    "有趣的例子：\n",
    "\n",
    "https://zhuanlan.zhihu.com/p/480831103\n",
    "\n",
    "\n",
    "\n",
    "图分类 (Graph classification) 指的是对于已知的图数据集, 基于一些结构图的属性, 分类整张图的任务. 因此, 我们需要嵌入整张图, 并且使它们在某些任务下是线性可分的.\n",
    "\n",
    "\n",
    "## TUDataset 数据集\n",
    "\n",
    "图分类中, 最常见的任务是 分子性质预测 (molecular property prediction), 其中一个分子被表达成一张图. 举个例子, 任务可以是推断一个分子是否抑制HIV病毒的复制. 多特蒙德工业大学收集了广泛的图分类数据集, 取名为TUDatasets. 在PyG中, 我们可以通过 torch_geometric.datasets.TUDataset 来获取这个数据集. \n",
    "\n",
    "这个数据集提供 188张不同的图, 我们的任务是分类每张图到两个类别中的一个.\n",
    "\n",
    "通过检查数据集的第一个图对象, 我们可以发现它有 17个节点 (每个节点有 7维的特征向量), 和 38条边 (平均节点出入度数 38/17=2.24), 每张图都有 一个图标签 y=[1]. 另外, 每条边还有额外的 4维边特征 (edge feature) edge_attr=[38, 4]. 但是, 为了让本教程足够简单, 我们不会使用这些额外特征.\n",
    "\n",
    "PyG 提供一些便利函数来帮助我们更好地处理图数据集, 例如, 我们可以 洗牌 (shuffle) 数据集, 并使用前150个图作为训练集, 其余的用作测试:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mHSP6-RBOqCE",
    "outputId": "47f23f5f-d41b-4f1a-f03c-c3b485c2e63f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Dataset: MUTAG(188):\n",
      "====================\n",
      "Number of graphs: 188\n",
      "Number of features: 7\n",
      "Number of classes: 2\n",
      "\n",
      "Data(edge_index=[2, 38], x=[17, 7], edge_attr=[38, 4], y=[1])\n",
      "=============================================================\n",
      "Number of nodes: 17\n",
      "Number of edges: 38\n",
      "Average node degree: 2.24\n",
      "Has isolated nodes: False\n",
      "Has self-loops: False\n",
      "Is undirected: True\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch_geometric.datasets import TUDataset #分子数据集：https://chrsmrrs.github.io/datasets/\n",
    "from torch_geometric.utils import to_networkx\n",
    "import networkx as nx\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "dataset = TUDataset(root='data/TUDataset', name='MUTAG')\n",
    "\n",
    "print()\n",
    "print(f'Dataset: {dataset}:')\n",
    "print('====================')\n",
    "print(f'Number of graphs: {len(dataset)}')\n",
    "print(f'Number of features: {dataset.num_features}')\n",
    "print(f'Number of classes: {dataset.num_classes}')\n",
    "\n",
    "data = dataset[0]  # Get the first graph object.\n",
    "\n",
    "print()\n",
    "print(data)\n",
    "print('=============================================================')\n",
    "\n",
    "# Gather some statistics about the first graph.\n",
    "print(f'Number of nodes: {data.num_nodes}')\n",
    "print(f'Number of edges: {data.num_edges}')\n",
    "print(f'Average node degree: {data.num_edges / data.num_nodes:.2f}')\n",
    "print(f'Has isolated nodes: {data.has_isolated_nodes()}')\n",
    "print(f'Has self-loops: {data.has_self_loops()}')\n",
    "print(f'Is undirected: {data.is_undirected()}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([1])\n",
      "tensor([0])\n",
      "tensor([0])\n",
      "tensor([1])\n",
      "tensor([0])\n"
     ]
    }
   ],
   "source": [
    "for i in dataset:\n",
    "    print(i.y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data(edge_index=[2, 20], x=[10, 7], edge_attr=[20, 4], y=[1])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 图可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAHiCAYAAAA597/kAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/T0lEQVR4nO3de3CUZaLn8d/bl3Snk849hISb3ATC/SIEVFBH5TYeb6COeAYRL1vFVJ06u1tL1fHU1m5tndrlVO1UTU0xZ5VRzDg6HtEZZwQTLwMcEQy3ACKgjoIgJJCkk869k/Rl/3hJAEEJkOR9u/v7qUpF+w3Jr0X98Tzv8z6PEYvFYgIAAJZyWB0AAABQyAAA2AKFDACADVDIAADYAIUMAIANUMgAANgAhQwAgA1QyAAA2ICrN18UjUZVVVUlv98vwzD6OxMAAAkjFoupublZRUVFcjh+eBzcq0KuqqrSsGHD+iwcAADJ5rvvvtPQoUN/8HqvCtnv9/d8s4yMjL5JBgBAEmhqatKwYcN6uvSH9KqQu6epMzIyKGQAAK7D1W75sqgLAAAboJABALABChkAABugkAEAsAEKGQAAG6CQAQCwAQoZAAAboJABALABChkAABugkAEAsAEKGQAAG6CQAQCwAQoZAAAboJABALABChkAABugkAEAsAEKGQAAG6CQAQCwAQoZAAAboJABALABChkAABugkAEAsAEKGQAAG6CQAQCwAQoZAAAboJABALABChkAABugkAEAsAEKGQAAG6CQAQCwAQoZAAAboJABALABl9UB4lpVlVRWJgUCUm6utHixVFRkdSoAQByikK9HOCytWyeVlkrNzZLDIUWj5msrV0pr10ou/tECAHqP1rge69ZJ69dLXq9UWHihkINB83VJev55SyMCAOIL95Cv1Zkz5sjY65VycswylszPOTnm66Wl5nQ2AAC9RCFfq/Jyc5o6K6vnpXAkcuF6VpZ5vaxswKMBAOIXhXytAgFzNHx+ZNzZ2amamhqFOjrM693XAgELQwIA4g2FfK1yc837xdGoJMmdkiKPx6NgQ4MikciFa7m5FgcFAMQTCvlaLVok+f3mAi5JhqSsrCzJMNTQ0KBYMGheX7zYwpAAgHhDIV+rIUPMR5tCIam+XopG5XQ4lJ2ZKUdTkzobG83rPI8MALgGPPZ0PdauNT+XlkrV1ZLDIU80qmhamj4ZO1bDHnxQN1ubEAAQZyjk6+Fymc8Zr1p1YaeuvDx5Fy5Uzccfa//mzXpuyBBlZmZanRQAECeMWCwWu9oXNTU1KTMzU42NjcrIyBiIXHGrvb1dL7zwgvx+v5588kk5nU6rIwEALNTbDuUech9LTU3VsmXLVFVVpY8++sjqOACAOEEh94OhQ4fqnnvuUUVFhb744gur4wAA4gCF3E/mzJmj8ePH65133lFDQ4PVcQAANkch9xPDMHT//fcrNTVVmzZtUjgctjoSAMDGKOR+5PV6tXz5ctXU1OiDDz6wOg4AwMYo5H5WVFSkhQsXau/evTpy5IjVcQAANkUhD4BZs2Zp4sSJ+stf/qIAh04AAK6AQh4AhmHovvvuU3p6ut566y11dXVZHQkAYDMU8gDxeDxavny56urqVF5ebnUcAIDNUMgDaPDgwVq0aJEqKyt1+PBhq+MAAGyEQh5gM2bM0JQpU/Tuu++qrq7O6jgAAJugkAeYYRhaunSpMjMztWnTJu4nAwAkUciWSElJ0fLly1VfX6/33nvP6jgAABugkC0yaNAgLV26VAcPHtTBgwetjgMAsBiFbKFp06Zp2rRp2rJli2pqaqyOAwCwEIVssSVLlignJ0ebNm1SZ2en1XEAABahkC3mdru1fPlyNTY2avPmzYrFYlZHAgBYgEK2gby8PN133306fPiwKisrrY4DALAAhWwTkydP1owZM1RWVqazZ89aHQcAMMDiv5CrqqSXXpL+9V/Nz1VVVie6bosXL1Z+fr42bdqkjo4Oq+MAAAaQEevFTcumpiZlZmaqsbFRGRkZA5Hr6sJhad06qbRUam6WHA4pGpX8fmnlSmntWsnlsjrlNauvr9cLL7ygsWPH6uGHH5ZhGFZHAgDcgN52aPyOkNetk9avN4u5sPDCRzhsvr5undUJr0tOTo7+7u/+TkeOHNG+ffusjgMAGCDxWchnzpgjY69XyskxR8eS+Tknx3y9tDRup68nTpyoW265Re+//76q4vQ9AACuTfzN6UpSebk5TV1Y2PNSMBhUqKNDDodDDklpDQ368n//b9Xdf79SU1Pl8/mUmpp62V87nU7r3sePuPfee3XmzBlt2rRJz913n7zbtkmBgJSbKy1eLBUVWR0RANCH4rOQAwFzNOy4MMD3er1yOp2KRqOKRqOKSeqoqtKRI0fU3t7+g4ukUlJSLilpn88nr9f7gwXefb2/7+26XC4te+ABfbZihbr+z/+RR5LRfZ983bq4vk8OALhcfP7fPDfXLKZotKeUvV6vvF6veT0aldraNGfJEs1ZvVqSFIlEFAqF1NbWpvb2drW3t1/216FQSC0tLaqtre25Fg6HL/vxhmHI6/VeVtY/NhL3+Xxyu93XVOTZL76oeQcPqrmrS62DBind7zffWzBo3ieXpOefv6F/lAAAe4jPVdZnzkh33mku4MrJufx6fb05cty+/Yandru6un6wwLv/+krXrvSP1el0XrGsv/+az+dTWjCo7IcfliMaVZPLpdbWVuXl5iolJaXP3yMAoP/0tkPjc4Q8ZIg5Zbt+vVlMWVkXHnsKBqVQSFqzpk+Kyu12y+12X9MfRGKxmDo6OnpV5OfOnev561Ao1PM9pldW6s5z59Scnm5OVUtqCAblT09XamqqjKwsqbpaKiuTzs8CAADiV3wWsmTeP5XM1dTV1Zc+h7xmzYXrFuie0vZ6vcrOzu71r4tGoz3T6q5f/lLeTz+VMztb0VhMkUhEkXBYwWBQTc3NSvP5lGYYcgQC/fhOAAADJX4L2eUy75+uWmWOEgMBKS9PWrQobqdwHQ6HfD6ffD6fNHq05HTK7fVesnitKxxWa2urWpqbpeZmfX3qlIrOnVNBQYGFyQEANyo+7yEng6vcJ48GAuoIh/W7p57SWYdDo0aNUklJicaMGcPuXgBgI4m/U1ei675PHgqZ98mjUfP1aFSqr5ejo0Op/+k/6en//t/10EMPKRQK6fXXX9f69eu1b98+zlYGgDjDCNnOrmG/7lgspu+++04VFRX64osv5PF4NHPmTM2ePZvfMwCwUG87lEKOB1VV13SfvKGhQXv27FFlZaXC4bAmTpyokpISFcXpvXUAiGcUMtTR0aEDBw5o9+7dCgaDGj58uEpKSjRu3Dg5HNytAICBQCGjRzQa1ZdffqmKigqdOnVKWVlZmjNnjqZPny6Px2N1PABIaBQyrqiqqkoVFRU6cuSIXC6Xpk+frjlz5lzT89IAgN6jkPGjmpqatHfvXu3fv1+hUEjjx49XSUmJhg0bxmNTANCHKGT0SldXlw4dOqSKigoFAgEVFRWppKRExcXFtj2aEgDiCYWMaxKLxfT111+roqJCx48fl9/v1+zZszVz5kylpqZaHQ8A4haFjOt27tw5VVRU6PDhwzIMQ1OnTlVJSYny8vKsjgYAcYdCxg1rbW3Vvn37tHfvXrW2tmrs2LEqKSnRyJEjuc8MAL1EIaPPhMNhff7556qoqNC5c+c0aNAglZSUaPLkyXK54vd8EgAYCBQy+lwsFtO3336riooKffXVV0pLS9OsWbM0a9YspaenWx0PAGyJQka/CgQCqqio0KFDhxSNRjV58mSVlJRwDCQAfA+FjAHR3t6u/fv3a8+ePWpubtbIkSNVUlKisWPHcp85WV2893purrR4cdyeUQ70BQoZAyoSiejYsWOqqKjQmTNnlJubqzlz5mjq1KlKSUmxOh4GwjWcTgYkEwoZlojFYjp9+rQqKip07NgxjoFMJv/yL9L69ZLXK2VlXSjkYNA813vNGun5561OCQw4ChmWCwaD2r17tw4cOKDOzs6eYyCHDBlidTT0tTNnpDvvNEfJOTmXX6+vN0fH27czfY2k09sO5Qw+9JusrCwtXLhQ//iP/6h7771Xp0+f1m9/+1u9/PLLOnr0qKLRqNUR0VfKyxVrblbY71eoo0MtLS1qD4UU6f49zsoyp7HLyiyNCdgZN3TQ7zwej0pKSjR79uyeYyA3bdqkzMzMnmMgvV6v1THRS7FYTE1NTaqtrVVNTY1qa2tVWFamCa2taq6tlSQZDoccDodi0agyMjOVmpoqw+EwF3oBuCIKGQPG4XBowoQJmjBhgqqqqrR792599NFH2r59O8dA2lAsFlNzc/Mlxdv90dHRIUlyu93Ky8vT8MGD5XG75c7OlislRU6nU9FoVE1NTQo2NKitpUU54bAcubkWvyvAvriHDEs1Nzdrz5492r9/v9rb23uOgRw+fDiPTQ2QWCymlpaWKxZvKBSSJLlcLuXl5WnQoEHKz89Xfn6+Bg0apKysLPP36UfuIYc6OhSqrlZE0pf/7/9p9gMPsMMbkgqLuhBXuo+B3L17t+rq6lRYWKiSkhJNnDiRYyD7UGtra0/pXvy5u3idTucPFq/DcZUlJz+yyjoWCun4okV6feRIZWdna+nSpRo5cmT/v2HABihkxKUrHQN5yy23aObMmfL5fJd+MRtQ/KC2trbLire2tlZtbW2SzOLNzc29rHizs7OvXrw/pBfPIdfU12vLli06deqUpk6dqnvuuUdpaWl9+M4B+6GQEfdqampUUVGhzz777NJjILOy4nsDij78g0R7e/sVi7e1tVWSed/+SsWbk5Nz/cV7NRe/v7w8adGiS95fLBbTgQMH9OGHH8owDN1zzz2aNm0atyiQsChkJIzvHwN5/+efa9LHH8uZliYjnjaguIGdrEKh0BWLt6WlRZJkGMYPFq9dp/xbW1v1wQcf6LPPPtOIESO0dOlS5efnWx0L6HMUMhJOOBzWl1u3asjf/72inZ0KZ2QoLS1NnpQUGQ6HDElqaJDhcim2bZscQ4daHflSvdjJquO//tee0r24eJubmyWZxZuTk3NZ8ebm5tq2eK/m+PHj2rJli4LBoG699VbdfvvtcrvdVscC+gyFjMT00kuK/fM/qzMnRy3t7ers7FTs4g1GYjH5W1q07a67dHDmTDkcDjmdzss+X+m13ly73u/nrqnR0J//XEYkolhWlmQYisViioTD6gqH5QgGFY7F9PLPf66WjAwZhqHs7OwrFm8irlAOh8PasWOHPvnkE2VlZWnp0qUaNWqU1bGAPtHbDk28/7KR2AIBGQ6HPKmp8qSmqiscVlNjozo6OmQ4HPJ6vfJEIpp5000qWrJEkUhE0Wj0ks9Xei0ajV52vaurq9e/9uJrV/oz7vTKSuXX1ak5PV2qq5MkpaSkqLOzU06XS26fT2nNzXrE75f7ueeUm5ubVKNEl8ulO++8U5MnT9bmzZv16quvavLkyVq4cCGLvpA0KGTEl9xcc5o3GpUcDrldLuXm5iocDqulpUVtbW1yh0Jq9/k0ZcoUS06aisVilxW165e/lHvPHqUWFChmfpFkGOZIunsxU1eXhvl80uDBA57ZLvLy8rRy5UodOnRIH3zwgf72t7/pnnvu0fTp01n0hYTHXtaIL4sWmYuggsFLXna5XMrKylJBSoqUnq53w2H96le/0s6dO9XZ2TmgEQ3DkMvlUkpKilJTU5Weni7vkCFySnKd/0OE2+2W2+W6UMbdf8hgJysZhqFp06bpF7/4hcaPH693331XGzduVE1NjdXRgH7FPWTEn14sjgquWaMdO3bo4MGD8nq9mjt3rmbPnm3d2cychnTdvv32W23evFkNDQ2aN2+e5s+fn1TT+Yh/LOpC4rqGx4eCwaA++eQTHThwQB6Pp6eYPR7PwOfmvODrFg6HtXPnTu3YsUMZGRlasmSJxowZY3UsoFcoZCS+q2xAcbHGxsaeYk5JSVFJSYnmzJkzsMV8A88hwxQIBLRlyxadOHFCkyZN0sKFC5Wenm51LOBHUcjAFTQ1NemTTz5RZWWl3G53TzEP6PGP1/AHCVwuFovps88+0wcffKBIJKK7775bM2fOZNEXbItCBn5EU1OTdu7cqf3798vtdmvOnDkqKSnhXOY40tbWpo8++kgHDhzQ0KFD9dOf/lQFBQVWxwIuQyEDvdDc3NxTzE6ns6eYU1NTrY6GXjp58qQ2b96sQCCguXPnasGCBdYt3gOugEIGrkFLS4t27typffv2yel0avbs2Zo7dy7FHCcikYh27typjz/+WH6/X0uWLNHYsWOtjgVIopCB69LS0qJdu3Zp3759Mgyjp5gvO/oRtlR//njH48ePq7i4WIsWLZLf77c6FpIchQzcgNbWVu3atUt79+6lmONMLBbT559/rvfff1/hcFh33XWXZs2a1X/HTQJXQSEDfaC1tVWffvqp9uzZI0k9xcz+yvbX3t6ujz76SJWVlRoyZIh++tOfanD3tqR9eCY1cDUUMtCH2traeoo5Fovplltu0bx58yjmOHDq1Clt3rxZdXV1mnvLLbpz7165fv97ngXHgKGQgX7Q1tamiooK7d69W7FYTLNmzdK8efPYnMLmIpGIPv30U3X9j/+hW/buVUpGhlLy89ktDQOCQgb6UXt7e08xRyIRzZo1S7feeivFbGdnzigyf77ampvV7HLJm5qqzIwMOZ1O8zr7iaOf9LZDWeUAXIfU1FTdeeed+od/+AfdeuutOnDggH71q1+pvLxczc3NVsfDlZSXy9nWpvShQ5Wdna3Ojg61tLZeuJ6VZU5jl5VZFhHJjZslwA1ITU3VHXfcoZKSEu3evVsVFRXat2+fZs6cqVtvvZUZJTsJBCSHQ4bDodTUVIUjEbW0tCiz+/fI4TA/AgFrcyJpUchAH/B6vVqwYIHmzJmjPXv26NNPP9X+/fs1Y8YM3XbbbRSzHeTmXjh32uEw976++I4dZ1LDYtxDBvpBR0dHTzF3dnZq+vTpuu2225SZmWl1tOT1vTOpW9va1BgMqrCoSIbEPWT0m952KCNkoB94PB7dfvvtmj17tvbu3atdu3apsrKyp5izsrKsjph8hgwxH21av16qr5fRvd/191dZU8awCCNkYAB0dnb2FHMoFNK0adN0++23U8wD7aIzqbsaGtQeCsmfliYjI4PnkNFveOwJsKHvF/PUqVN1++23Kzs72+poyaWqSlUvvaQjH3+sBQ8/rJS/+ztGxug3TFkDNpSSkqJbb71Vt9xyi/bt26ddu3bp4MGDPcWck5NjdcTkUFSk5kce0a5oVHOfeEIpPD8OG6CQAQukpKRo3rx5lxTzoUOHNGXKFM2fP59iHgCu81PTkUjE4iSAiUIGLOR2uzV37lzNmjVL+/fv186dO/XZZ59p8uTJmj9/vnJ5BKffdO/QRSHDLihkwAbcbrdKSkouKebDhw9r0qRJmj9/vvLy8qyOmHC6CzkcDlucBDBRyICNuFwuzZkzRzNnzlRlZaV27typ9evX9xRzfn6+1RETBiNk2A2FDNiQy+XS7NmzNWPGDB04cECffPKJfvOb32jixImaP3++Bg0adPkv4ozfa0Ihw24oZMDGXC6XbrnlFk2fPl0HDx7UJ598on/7t39TcXGxFixYYBbzRc/WXnLG77p1PFv7I1jUBbvhv1IgDrhcLs2aNeuKxbyoslL+0lLJ65UKCy8943f9evMbcMbvZRghw244fhGII06nUzNnztQvfvEL3XfffWo8elSdL76olkhEXX6/WcaS+Tknxyzp0lJzOhuXYFEX7IZCBuKQ0+nUjBkz9FRRkTIMQ60pKaqtrVV9Q4M6OzvVs/0eZ/z+IEbIsBumrIE45mhokMPj0aCCArW3tamlpUUNwaAk80jIVK9XbodDBmf8XoZ7yLAbChmIZ+fP+DWiUfl8PqX6fOrs6FB7KKT2tja1Njcrs7VVx6urlXHypIYNGyaHg4kxiREy7IdCBuLZokXmaupgUMrJkSHz6EePx6NYZqbCNTXq9Pu1KyNDZ195Renp6Ro/fryKi4s1YsSIpC7n7vdOIcMuKGQgnn3vjF9lZfWssjaCQbnDYbnXrNGz//RPOn36tI4ePapjx45p37598vl8GjdunIqLizVy5MieEWOyMAxDTqeTRV2wDQoZiHdr15qfS0ul6uoLjz35/dKaNdLatTIMQ8OGDdOwYcN07733qrq6uqecDxw4IK/Xq3HjxmnChAkaPXp0z/3VROd0OhkhwzaS4786IJG5XOZzxqtWXdipKy/PnM6+wk5dhmGoqKhIRUVF+slPfqJz587p2LFjOnr0qA4dOqSUlBTdfPPNKi4u1pgxY+R2uy14UwODQoadUMhAoigqklavvqZfYhiGBg8erMGDB+vOO+9UbW1tz8j5zTfflNvt1tixYzVhwgSNHTtWHo+nn8Jbg0KGnVDIAHrk5+drwYIFWrBggerr63vK+e2335bT6dSYMWM0YcIEjRs3Tl6v1+q4N4x7yLATChnAFeXk5Oi2227TbbfdpmAw2DOt/c4778jhcGjUqFGaMGGCxo8fL5/PZ3Xc6+JyuRghwzYoZABXlZWVpblz52ru3LlqamrSsWPHdOzYMb377rvavHmzbrrpJhUXF2v8+PFKT0+3Om6vMWUNO6GQAVyTjIwMzZkzR3PmzFFLS4u++OILHTt2TO+99562bNmiESNGaMKECZowYYIyMjKsjvujKGTYCYUM4Lqlp6dr1qxZmjVrltra2vTll1/q2LFj+uCDD1ReXq6hQ4dqwoQJKi4uVlZWltVxL0Mhw04oZAB9wufzafr06Zo+fbpCoZC++uorHT16VFu3btWHH36owsJCFRcXa8KECcrNzbU6riQKGfZCIQPoc16vV1OmTNGUKVPU0dGhv/3tbzp27Jg+/vhj/fWvf1VBQUHPyDk/P9+ynCzqgp1QyAD6lcfj0aRJkzRp0iR1dXXp66+/1rFjx7Rr1y5t375deXl5PeVcUFAgwzAGLBuPPcFOKGQAA8btdvcs+AqHwzp+/LiOHj2qvXv3aseOHcrOzu4p56Kion4vZ6fTqc7Ozn79GUBvUcgALOFyuXTzzTfr5ptvViQS0YkTJ3Ts2DEdPHhQu3btUmZmZk95Dxs2rF/KmXvIsBMKGYDluncBGzNmjJYuXaqTJ0/q6NGj+vzzz1VRUaH09PSekfPw4cP77NhIChl2QiEDsBWHw6GRI0dq5MiRWrJkib777rueLTz37t0rn8/Xc6bzTTfddEPHRlLIsBMKGYBtGYah4cOHa/jw4Vq4cKGqqqp6yrmyslJer1fjx4/XhAkTNGrUqGs+NpJFXbATChlAXDAMQ0OGDNGQIUN099136+zZsz37ax88eFAej6fn2MjRo0df/djIqiqN+OgjZR4/LqWkSIsXX/G4SmCgGLFYLHa1L2pqalJmZqYaGxttvxUegOQSi8UuOTaypqZGbrdbN998c8+xkSkpKRd+QTgsrVsnlZaqIxBQVzisdJ9P8vullSultWvNM6aBPtLbDqWQASSUQCDQU87V1dVyuVw9x0befPPN8v7f/yutXy95vWp2OtUaCmlwfr4UDEqhkLRmjfT881a/DSQQChlA0mtoaOg5mer06dPKam3VqldflcfplHvQILW2tam1pUWDBw82f0F9vTk63r6d6Wv0md52KPMyABJWdna25s2bp3nz5qmxsVGBf/1XudrbFUhNlc6dk9PpVDQWU0ySIUlZWVJ1tVRWJq1ebW14JJ2+eZgPAGwuMzNTozIz5UtLU0FhoTIzMuRwOOQwDAWDQcViMcnhMD8CAavjIgkxQgaQPHJzpWhUTklpaWlKS0tTW3u7GoNB1YXDysnKkjMaNb8OGGCMkAEkj0WLzNXUwWDPS77UVOXm5Skaiaj5u+/UlZpqPgIFDDAKGUDyGDLEfLQpFDIXcEWjkqQUp1P5LpfckYg+GTVKB2tqLA6KZMSUNYDksnat+bm01FzA5XBI0agcfr+8/+W/qGXSJH385z/r3Llzuueee/ps32zganjsCUByqqoyV1MHAlJenjmdXVSkWCymvXv3qry8XCNHjtSyZcuUmppqdVrEMZ5DBoAbcOLECW3atEmpqal67LHHlJ+fb3UkxKnedihzMQBwBSNHjtQzzzwjl8ul3/72t/ryyy+tjoQERyEDwA/Izs7W6tWrNWrUKL3xxhvasWOHejGpCFwXChkAfkRKSooeeeQRLViwQFu3btXbb7+trq4uq2MhAVHIAHAVhmHojjvu0PLly/XVV1/p5ZdfVmNjo9WxkGAoZADopeLiYq1evVqhUEgvvviiTp48aXUkJBAKGQCuQUFBgZ555hkNGjRIv/vd77R//36rIyFBUMgAcI18Pp+eeOIJzZw5U5s3b9aWLVsUiUSsjoU4x05dAHAdnE6nlixZooKCAr333nuqq6vT8uXL5fP5rI6GOMUIGQBuwMyZM7Vy5UrV1NRow4YNOnfunNWREKcoZAC4QcOHD9ezzz4rr9erl156SUePHrU6EuIQhQwAfSAzM1NPPfWUbr75Zm3atEnbtm1jExFcE+4hA0Afcbvdevjhh1VQUKCtW7eqpqZGDzzwgDwej9XREAcYIQNAHzIMQ7fffrsee+wxHT9+XC+//LIaGhqsjoU4QCEDQD8YN26cnn76aYXDYW3YsEEnTpywOhJsjkIGgH6Sn5+vp59+WkVFRXr11Ve1Z88e7ivjB1HIANCPUlNT9fjjj2vOnDkqKyvTu+++q3A4bHUs2BCLugCgnzkcDi1cuFAFBQXavHmz6urq9Mgjjyg9Pd3qaLARRsgAMECmTZumJ598Ug0NDdqwYYOqqqqsjgQboZABYAANHTpUzz77rPx+vzZu3KjDhw9bHQk2QSEDwADz+/168sknVVxcrD/+8Y/66KOPFI1GrY4Fi3EPGQAs4HK59MADD2jw4MH68MMPVVNTo4ceekher9fqaLAII2QAsIhhGJo7d64ef/xxfffdd3rppZcUCASsjgWLUMgAYLExY8bo6aefliRt2LBBX3/9tcWJYAUKGQBsIDc3V6tXr9bw4cP1+uuva9euXWwikmQoZACwCa/Xq8cee0y33nqrPvzwQ73zzjvq6uqyOhYGCIu6AMBGHA6HfvKTn6igoEB//vOfVVdXp0cffVQZGRlWR0M/Y4QMADY0adIkPfXUU2ppadGGDRt0+vRpqyOhn1HIAGBThYWFeuaZZ5Sdna1XXnlFBw8etDoS+hGFDAA2lp6erp///OeaMmWK/vznP6u8vJxNRBIU95ABwOZcLpfuu+8+DR48WOXl5aqtrdWyZcuUmppqdTT0IUbIABAHDMPQ7Nmz9fd///eqrq7Whg0bVFNTY3Us9CEKGQDiyMiRI/XMM8/I7XbrpZde0pdffml1JPQRChkA4kx2drZWr16tUaNG6Y033tDHH3/MJiIJgEIGgDiUkpKiRx55RAsWLNC2bdv01ltvqbOz0+pYuAEUMgDEKcMwdMcdd2j58uX629/+po0bNyoYDFodC9eJQgaAOFdcXKzVq1crFAppw4YNOnnypNWRcB0oZABIAAUFBXrmmWc0aNAg/e53v9O+ffusjoRrxHPIAJAgfD6fnnjiCb3//vvasmWLzp07p0WLFsnpdEpVVVJZmRQISLm50uLFUlGR1ZFxEQoZABKI0+nUkiVLVFBQoPfee091Z8/qZ6dOKeUPf5CamyWHQ4pGpXXrpJUrpbVrJRdVYAf8LgBAApo5c6by8/N18tln1fnpp3JkZclVWHihkINBaf1684uff97SrDBxDxkAEtRwp1Pzvv5aUbdbteGw2rsfi3I4pJwcyeuVSkvN6WxYjkIGgERVXi5na6vShw2Tx+tVU1OTgo2N6tlCJCvLnMYuK7MwJLpRyACQqAIByeGQw+lUht+vSDisttZWBQIBRaJRc6TscJhfB8tRyACQqHJzzfvF0ahkGJKkjMxMhcNh1dbWqiMUMq/l5locFBKFDACJa9Eiye83F3Cd53K5lJ+fL5fLpdaqKrW7XIotWmRdRvSgkAEgUQ0ZYj7aFArJaGiQzh9A4ZSUaxhKMwztGjtW/75jh0KhkLVZQSEDQEJbu1Zas0ZyueRvaZGrpkaqrpbhcsnzn/+zhv761zp58qRefPFFVVdXW502qRmxXpzZ1dTUpMzMTDU2NiojI2MgcgEA+lDTF19o+9q1mjdunPLGjzens8/v1NXQ0KBNmzappqZGS5Ys0YwZMyxOm1h626FsDAIAyaCoSAdmzNCExx9X3tixl1zKzs7WU089pbKyMr377rv67rvvtGTJErndbovCJicKGQAgl8ul++67T8OHD9fmzZtVXV2t5cuXK5cV2AOGe8gAkASM8489Xe0u5dSpU/X000+rq6tLGzZs0LFjxwYiHkQhAwC+p6CgQM8++6xGjx6tN998Ux988IEikYjVsRIehQwASaC3I+RuHo9Hy5Yt08KFC7V792797ne/U3Nzc39GTHoUMgDgigzDUElJiVauXKmGhga98MILOnHihNWxEhaFDABJ4FpHyBcbPny4nnvuOQ0aNEivvvqqduzYcV3fBz+OQgYAXFVaWpqeeOIJ3Xbbbdq6daveeOMNtbe3Wx0roVDIAJAEbmSE3M3hcOiuu+7S448/ru+++04vvviiqjhLuc9QyACAazJ27Fg9++yz8vl8evnll7Vv3z6msPsAhQwASaAvRsgXy8rK0qpVqzR9+nRt2bJF77zzjjo7O/vkeycrduoCAFwXl8ulpUuXavjw4Xr33Xd19uxZLV++XHl5eVZHi0uMkAEgCfT1CPlikydP1tNPP61IJKINGzboyJEjff4zkgGFDAC4YYMGDdIzzzyjsWPH6q233lJ5eTm7e10jpqwBIAn05wi5m8fj0cMPP6zhw4fr/fff15kzZ7R8+XKO7e0lRsgAgD5jGIZmz56tVatWqampSS+88IK++eYbq2PFBQoZAJLAQIyQLzZ06FA999xzKiws1O9//3v9x3/8B49GXQWFDADoFz6fT48//rgWLFig7du36/XXX1dbW5vVsWyLQgaAJDDQI+RuDodDd9xxh1asWKEzZ87ohRde0JkzZwY0Q7ygkAEA/W7MmDF67rnn5Pf79fLLL2vPnj1MYX8PhQwAScCqEfLFMjMztWrVKs2aNUtlZWX64x//yO5eF6GQAQADxul0avHixXr44Yf11VdfacOGDaqtrbU6li1QyACQBOwwQr7YpEmT9Mwzz8gwDG3YsEGHDx+2OpLlKGQAgCXy8vL09NNPa/z48frjH/+o9957T+Fw2OpYlmGnLgBIAnYbIXdLSUnRgw8+qOHDh6u8vFxVVVVatmyZsrKyrI424BghAwAsZRiGZs2apVWrVqmlpUUvvviivv76a6tjDTgKGQCSgF1HyBcbMmSInn32WQ0ZMkSvvfaatm3bpmg0anWsAcOUNQAkge5Ctrvu3b127Nih7du36/Tp03rooYeUlpYmVVVJZWVSICDl5kqLF0tFRVZH7jMUMgAkETuPkLsZhqH58+dr6NChevvtt/Xib36jVbW1ynrnHam5WXI4pGhUWrdOWrlSWrtWcsV/ncX/OwAAJKRRo0bpueee01crV8r5178q5PfLU1goo7uQg0Fp/Xrzi59/3tKsfYF7yACQJAzDiIsR8sUymps18/PP5UxLU30spobGRkVjMXOUnJMjeb1Saak5nR3nKGQAgH2Vl8toaZGvsFDZOTnqCIUUDAbV88eKrCxzGruszMKQfYMpawBIBlVVml5ZqYLaWunQofhZEBUImKNhh0OpXq+Una2G+np1pacrxe3uuaZAwOqkN4wRMgAksnBY+pd/ke64Q3f89a8qeOMN6Z//WbrjDvN1u++MlZtr3i8+//hTitstSRceh+q+lptrVcI+QyEDQCJbt85c+BQOq9nvV1denlRYaBbx+vXmdTtbtEjy+80FXDLPV5akaCRiXg8GzeuLF1uTrw9RyACQqM6cMRc8eb3mAqjuZ5HjaUHUkCHmo02hkFRfLyMWk+FwKBoOS/X15usrV8bH9PtVUMgAkKjKy80FT+f3hXa5XBdKWYqfBVFr10pr1pjPGldXy9/cLHddnfn3a9aY1xMAi7oAIFFdtCBKkiKRiGIXb0UZLwuiXC7zOeNVq6SyMh0rL5ezoEAz/umfEmJk3I1CBoBEdfGCKIdDDsMwn+HtFm8LooqKpNWrdTw9XV1dXZqRQGUsMWUNAInrewuiDIfj0o1B4nRBVFpamlpbW62O0ecoZABIVN9fECWZU9bRaFwviEpLS1NbW5vVMfocU9YAkMi6FzyVliqtvt7cB7q11RwZx+mCqO4RciwWi5tTrHqDQgaARHbRgqjP/+f/lLupSdPvuceczo6zkXG3tLQ0RSIRdXR0yOv1Wh2nz1DIAJAMiop0ZtEiBYNBTV+1yuo0NyQtLU2S1NramlCFzD1kAEgSKSkp6uzstDrGDbu4kBMJhQwASYJCtjcKGQCShMfjUUdHh9UxblhqaqoMw6CQAQDxKVFGyIZhyOfzUcgAgPiUkpKirq6uC0cXxrFE3ByEQgaAJOHxeCRJXV1dFie5cYm4OQiFDABJIiUlRZIS4j4yI2QAQNzqLuREuI9MIQMA4lb3lDWFbE8UMgAkiUSbsm5ra0uIBWrdKGQASBKJNmUtKaEWdlHIAJAkEmnK2ufzSUqs3booZABIEi6XS4ZhJMyUtUQhAwDikGEYCbNbF4UMAIhriVLIKSkpcrlc3EMGAMSnlJSUhJiyNgwj4R59opABIIl4PJ6EGCFLifcsMoUMAEkkUaasJQoZABDHKGT7opABIIl4PJ6EuIcsKeHORKaQASCJMEK2LwoZAJJIohVyZ2dnQpzvLFHIAJBUEuWxJynx9rOmkAEgiSTaY09S4uzWRSEDQBLpnrKOxWJWR7lhFDIAIG51H8GYCPddKWQAQNxKpCMYnU6nvF4vhQwAiD/dI+REWthFIQMA4k53ISfCCFmikAEAcSqRpqwlChkAEKcSbco6kbbPpJABIIkwZW1fLqsDAAAGTkpdnaZXViq7vl6aMkVavFgqKrI61nXLamvT2I8/VqyrS0ZeXly/HyPWi6fDm5qalJmZqcbGRmVkZAxELgBAXwqHpXXrpNJSNVdXK8Xrlcflkvx+aeVKae1ayRVHY7Tz76fjxRfVWV+vNL9fjljMlu+ntx3KlDUAJIN166T166VwWK2ZmerMyZEKC81iW7/evB5Pzr8fRyym5vR0RQcNiu/3IwoZABLfmTNSaank9Uo5OTIcDnPrTIdDyskxXy8tlaqqrE7aO997PzIMRaLR+H0/51HIAJDoysul5mYpK0uSucNV9OK7lVlZ5vWyMkviXbOL3k8sGpXT6bz0ery9n/MoZABIdIGAOXp0mP/Ldzqd6rp4lXX3tUDAooDX6Pz76YxEVF9fL8Ph6Fk9Lin+3s95FDIAJLrcXCkaNT8keVJS1NXVZU7zSheu5eZaGPIa5OYq0tWlQF2dnC6X8nJzZVx8Pd7ez3kUMgAkukWLzNXHwaAkyeP1SpI6uzcHCQbN64sXW5PvGp0YN07BaFRpXV3Kzc2Vw/G9Kouz99ONQgaARDdkiPkoUCgk1dfLKcnldqujvV2qrzdfX7kyLp7fPXr0qH6/bZtO33WX/C6XHA0NPSN/RaNx934uRiEDQDJYu1Zas8Z8Nre6WhktLXLV1Snmcpmvr11rdcKrOnjwoN566y0VFxdr0quvyrjo/fR8xNH7+T42BgGAZFJVJZWVKfDVV/rkiy906//6X8qbMsXqVFe1e/dulZeXa8aMGVq6dOmFaerz70eBgJSXZ07P22xk3NsOtcc2JgCAgVFUJK1erYyuLh1et06DWluVZ3WmHxGLxbRjxw5t27ZNc+fO1T333CPDuGgJ1/n3kwiYsgaAJOR2uzVixAgdP37c6ig/KBaL6cMPP9S2bdt05513Xl7GCYZCBoAkNXr0aH377bcKh8NWR7lMNBrV5s2b9emnn2rRokWaP39+QpexRCEDQNIaPXq0wuGwTp06ZXWUS0QiEf3pT3/SgQMHdP/992vOnDlWRxoQFDIAJKlBgwYpPT1d33zzjdVRenR1denNN9/U0aNHtWzZMk2bNs3qSAOGQgaAJGUYhkaPHm2bQu7o6NDrr7+u48eP62c/+5mKi4utjjSgKGQASGKjRo3SuXPn1NLSYmmO9vZ2vfrqq6qurtYTTzyhMWPGWJrHChQyACSx0aNHS5Klq62bm5v1yiuvqKGhQStXrtSIESMsy2IlChkAklhaWpoGDx5s2bR1MBjUxo0b1d7erieffFKFhYWW5LADChkAklz3feRebNzYp+rq6rRx40ZJ0qpVq5Sfnz+gP99uKGQASHKjR49Wa2urzp07N2A/8+zZs9q4caM8Ho9WrVql7OzsAfvZdkUhA0CSGzZsmFwu14BNW3/33Xd65ZVXlJWVpSeffFJ+v39Afq7dUcgAkORcLpduuummAVnY9c033+jVV1/V4MGD9fOf/1w+n6/ff2a8oJABABo9erROnjyprq6ufvsZx44d0x/+8AfddNNNWrFihTweT7/9rHhEIQMANHr0aEUiEZ08ebJfvv+hQ4e0adMmjRs3To8++qjcbne//Jx4RiEDAJSXl6eMjIx+uY+8d+9evfPOO5o2bZoefvhhOZ3OPv8ZiYDzkAEAMgxDo0aN6vNC3rFjh7Zu3aqSkhLde++9CX9i041ghAwAkGROW9fW1qqpqemGv1csFtNHH32krVu3asGCBZRxL1DIAABJ5r7Wkm54lByLxbRlyxbt3LlTCxcu1B133EEZ9wKFDACQJPl8PhUVFd3Q40/dZxnv379f9913n0pKSvowYWKjkAEAPW5kG81wOKxNmzbpyJEjWrZsmWbMmNEPCRMXhQwA6DF69Gi1t7erurr6mn5dZ2enXn/9dX3zzTd67LHHNHHixH5KmLgoZABAj6FDhyolJeWa7iN3n2V85swZrVixQmPHju3HhImLQgYA9HA6nde0jWZLS4tKS0sVCAS0cuVK3XTTTf0bMIFRyACAS4wePVqnTp1SZ2fnj35dY2OjNm7cqNbWVj355JMqKioaoISJiUIGAFxi9OjRikaj+vbbb3/wawKBgF5++WVFo1GtWrVKgwYNGriACYpCBgBcIicnR1lZWT94H/ncuXPauHGjUlJStGrVKuXk5AxwwsTE1pkAgEv82Daap0+f1muvvabs7GytWLFCaWlpFiRMTBQyAOAyo0eP1lfbt6vt17+Wr71dys3VyeJivbZtmwoLC/Wzn/1MXq/X6pgJhUIGAFwqHNbYTZtU8MorchmG5PEo3Nmp9FhMi++8U5P+8R/lpoz7HPeQAQCXWrdO7hdekNvhUIvfr7asLNW43fI4nZq2c6fcv/yl1QkTEoUMALjgzBmptFTyemXk5Kijs1PBhgb5fD6lDRsmw+s1r1dVWZ004VDIAIALysul5mYpK0uSZDgc8qWlKTMrS4Zkvt7cLJWVWRgyMVHIAIALAgHJ4VCoq0vNLS3yer3KzMxUz+GJDof5EQhYmTIhUcgAgAtycxUJh9UQCFxexpIUjZofublWJUxYrLIGAPRomDNHsWhUadGo0gsLLy1jSQoGJb9fWrzYgnSJjREyAECS1Nraqle3btWXc+Yo3eWSo6HBHA1L5uf6eikUklaulNi3us9RyACAnvOMu7q6NOGVV+T4xS8kl0uqrr7w4XJJa9ZIa9daHTchMWUNAEkuEolo06ZNqqur06pVq5SVlyc9/7y0apW5mjoQkPLypEWLGBn3IwoZAJJYLBbT5s2bdfz4ca1YsUKDBw++cLGoSFq92rpwSYYpawBIYtu3b9fBgwd1//33a9SoUVbHSWoUMgAkqf379+vjjz/W3XffrSlTplgdJ+lRyACQhL788ktt2bJFs2fP1rx586yOA1HIAJB0Tp8+rbfeekvjx4/XwoULZRiXPW0MC1DIAJBEAoGAXn/9dRUVFemhhx6Sw0EN2AW/EwCQJFpaWvT73/9e6enpeuyxx+Ry8aCNnVDIAJAEOjo69NprrykSiWjFihVKTU21OhK+h0IGgATXvfFHQ0ODVqxYoczMTKsj4QooZABIYLFYTH/5y1904sQJPfrooyooKLA6En4AhQwACWzr1q367LPP9OCDD2rkyJFWx8GPoJABIEHt2bNHn3zyie69915NmjTJ6ji4CgoZABLQsWPHVFZWppKSEs2dO9fqOOgFChkAEsypU6f09ttva+LEibr33nutjoNeopABIIHU1tbqD3/4g4YNG6YHHniAXbjiCIUMAAmiublZr732mjIyMvToo4+y8UecoZABIAGEQiG99tprisViWrFihbxer9WRcI0oZACIc5FIRG+++aYaGxu1YsUKZWRkWB0J14FCBoA4FovF9Oc//1mnTp3SY489pkGDBlkdCdeJQgaAOPbRRx/p8OHDevDBBzVixAir4+AGUMgAEKcqKiq0a9cuLVq0SBMnTrQ6Dm4QhQwAcejIkSN6//33NW/ePM2ZM8fqOOgDFDIAxJlvv/1Wf/rTnzR58mTdfffdVsdBH6GQASCO1NTU6I033tDw4cN1//33s/FHAqGQASBONDU16bXXXlNWVpYeffRROZ1OqyOhD1HIABAHujf+MAxDK1askMfjsToS+hiFDAA2Fw6H9cYbb6ipqUkrVqyQ3++3OhL6AYUMADYWi8X0zjvv6MyZM/rZz36m/Px8qyOhn1DIAGBTsVhM77//vo4cOaKHHnpIw4cPtzoS+hGFDAA29emnn2r37t1avHixJkyYYHUc9DMKGQBs6PDhw/rwww912223afbs2VbHwQCgkAHAZk6cOKF33nlHU6dO1V133WV1HAwQChkAbOTs2bP693//d40cOVL33XcfG38kEQoZAGwiGAzqtddeU3Z2tpYvX87GH0nGZXUAAEhKVVVSWZkUCEi5uWq/4w69Vl4ul8vFxh9JikIGgIEUDkvr1kmlpVJzs+RwKBaNqjMa1eRp01RcWqr09HSrU8ICTFkDwEBat05av94s5sJCxQoL1eD1KtrVpXmHDinvpZesTgiLUMgAMFDOnDFHxl6vlJOjmMOhpsZGhTo7lVpUJJfPZ16vqrI6KSxAIQPAQCkvl5qbFcvKUnsopEBdndrb25WZmSmv1ytlZZnT2GVlVieFBbiHDAADpO30aTm6uhSsrVU0EpHT5VI0GlVqaqr5BQ6H+REIWBsUlqCQAaAfRSIRffnll6qsrFRGZaXu6uxUamamfOnpisZiCtTVKRKJyOFySdGo+ZGba3VsWIBCBoB+UF9fr8rKSh08eFCtra0aOnSoRq9ZI9/Ro3JEIpLbrXA4LEmKRiKSyyUFg5LfLy1ebG14WIJCBoA+EolE9MUXX6iyslLHjx+Xx+PRlClTNHPmTBUUFJhf9OST5irr+no5MjPNXxcOS62tUigkrVkjFRVZ9yZgGQoZAG7Q90fDw4YN0/3336+JEyfK7XZf+sVr15qfS0vlOHtW/tZWuTo6zGnqNWsuXEfSMWKxWOxqX9TU1KTMzEw1NjYqIyNjIHIBgK11j4b379+vEydOyOv19oyGBw0adPVvcH6nrk83b1baiBGa8t/+GyPjBNXbDmWEDADXIBAI9IyG29raNHz4cD3wwAMqLi6+fDT8Y4qKpNWr9Y3HI7fbrSmUcdKjkAHgKsLhcM9o+Ntvv5XX69XUqVM1c+ZM5efn39D39vv9qqmp6aOkiGcUMgD8gEAgoP379+vQoUM9o+EHH3xQEyZMuLbR8I/IyMjQN9980yffC/GNQgaAi4TDYR07dkyVlZX69ttvlZqaqqlTp2rGjBk3PBq+Er/fr5aWFkWjUTkcbJ6YzChkAJBUV1fXMxpub2/XiBEj9NBDD2nChAlyufrvf5UZGRmKxWJqaWlh0WySo5ABJK3u0fD+/ft18uRJpaamatq0aZoxY4by8vIGJEN3CTc3N1PISY5CBpB0amtrVVlZ2TMavummm/Twww9r/Pjx/ToavhK/3y/JfDRmyJAhA/qzYS8UMoCkEA6HdfToUe3fv1+nTp2Sz+fTtGnTNHPmTOVauHe0z+eT0+lUU1OTZRlgDxQygPh0fmMNBQLmLleLF19xY43a2tqee8OhUEgjR460bDR8JYZhyO/3q7m52eoosJj1/zYCwLUIh6V166TSUvPsYIfDPCFp3Tpp5Upp7Vp1xWI6evSoKisre0bDM2bM0IwZMywdDf8QChkShQwg3qxbZx7O4PVKhYUXCjkYVOTXv9Y3X32lPxUX94yGly1bpnHjxtliNPxDMjIymLIGhQwgjpw5Y46MvV4pJ0eSFI3FFOroUFs0KkdHh/Lee09zFy/WpHvvVc75r7E7v9+vs2fPWh0DFqOQAcSP8nJzmrqwUJIUCoXUEAwqFo3K4/EotbBQ3oYGzW9t7SnseOD3+9XU1KRYLCbDMKyOA4tQyADiRyBgTlGf39HK5XYrzeeTz+e7MCXd2Gh+XRzJyMhQV1eXOjo65PV6rY4Di7BPG4D4kZtr3i+ORiVJLqdTGRkZF8q4+5oNF279mO4NQbiPnNwoZADxY9Eiye+XgsErXw8GzeuLFw9kqhvWvTkIK62TG4UMIH4MGWI+2hQKSfX1PSNlRaPm34dC5vU4O1v44t26kLy4hwwgvqxda34uLZWqqy889uT3S2vWXLgeR1wul3w+HyPkJEchA4gvLpf0/PPSqlUXdurKyzOns+NsZHyx7pXWSF4UMoD4VFQkrV5tdYo+k5GRwQg5yXEPGQBsgO0zQSEDgA2wfSYoZACwAb/fr9bWVkUiEaujwCIUMgDYQPfmIExbJy8KGQBsgM1BQCEDgA2wfSYoZACwAa/XK5fLxQg5ifEcMgDYgFFdrTmff66cEyekI0fM/bjjeKMTXDsjFovFrvZFTU1NyszMVGNjY8+0CgCgD4TD0rp1Ummp2mpqZDgcSvV4zK1AV640twJ1MXaKZ73tUKasAcBK69ZJ69dL4bA6cnLUmpEhFRaaRb1+vXkdSYFCBgCrnDljHpLh9Uo5OXK6XIpEo+aBGTk55uulpVJVldVJMQAoZACwSnm51NwsZWVJkpxOpySp5z5iVpZ5vazMinQYYNyYAACrBALmaNhhjo3S0tKUlpZ24Xr3tUDAooAYSIyQAcAqubnmWc7R6JWvd1/LzR3YXLAEhQwAVlm0yFxNHQxe+XowaF5fvHggU8EiFDIAWGXIEPPRplBIqq+/MFKORs2/D4XM6zyPnBS4hwwAVlq71vxcWipVV5v3jKNRc2S8Zs2F60h4bAwCAHZQVWWupg4EpLw8czqbkXFC6G2HMkIGADsoKpJWr7Y6BSzEPWQAAGyAQgYAwAYoZAAAbIBCBgDABihkAABsgEIGAMAGKGQAAGyAQgYAwAYoZAAAbIBCBgDABihkAABsgEIGAMAGKGQAAGyAQgYAwAYoZAAAbIBCBgDABihkAABsgEIGAMAGKGQAAGyAQgYAwAYoZAAAbIBCBgDABihkAABsgEIGAMAGKGQAAGyAQgYAwAYoZAAAbIBCBgDABihkAABsgEIGAMAGKGQAAGyAQgYAwAZcvfmiWCwmSWpqaurXMAAAJJru7uzu0h/Sq0Jubm6WJA0bNuwGYwEAkJyam5uVmZn5g9eN2NUqW1I0GlVVVZX8fr8Mw+jTgAAAJLJYLKbm5mYVFRXJ4fjhO8W9KmQAANC/WNQFAIANUMgAANgAhQwAgA1QyAAA2ACFDACADVDIAADYAIUMAIAN/H9mX3+r22UUBAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "TUNet=to_networkx(dataset[20]) # type : networkx.classes.graph.Graph\n",
    "TUNet=TUNet.to_undirected() # 转化为无向图\n",
    "pos=nx.spring_layout(TUNet) # 网络图中节点的布局方式\n",
    "plt.figure(figsize=(6,6))\n",
    "nx.draw_networkx_nodes(TUNet,pos,node_size=40,node_color=\"red\",alpha=0.8)\n",
    "nx.draw_networkx_edges(TUNet,pos,width=1,edge_color=\"gray\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据集的划分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "j11WiUr-PRH_",
    "outputId": "5f84abf9-05bc-4c62-95f1-5f80b603b0f0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training graphs: 150\n",
      "Number of test graphs: 38\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(12345)\n",
    "dataset = dataset.shuffle()\n",
    "\n",
    "train_dataset = dataset[:150]\n",
    "test_dataset = dataset[150:]\n",
    "\n",
    "print(f'Number of training graphs: {len(train_dataset)}')\n",
    "print(f'Number of test graphs: {len(test_dataset)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qeORu4Zrs8Zy"
   },
   "source": [
    "## 图的小批量 (mini-batching) 训练\n",
    "\n",
    "既然在图分类数据集中的单张图都比较小, 我们可以在输入到 GNN 之前, 将这些图 \"捆\" 成多个批次进行训练, 这样可以保证充分 GPU 的利用率. 在图像或语言领域, 这一过程通常通过将每个样例 缩放 (rescaling) 或 填充 (padding) 为一组形状相等的数据来实现 (这样输入数据就多了一个额外的维度). 这个维度的长度等于在小批量中的样本个数, 我们通常称其为 batch_size.\n",
    "\n",
    "但是, 对于GNN来说, 上述的两种方法都不可行, 或者说可能会导致不必要的内存消耗. 因此, PyG使用另一种方法优化, 以取得高效的样本之间的并行. 在本例中, 邻接矩阵 (adjacency matrix) 以对角阵的形式堆积, 也就是多个子图构建一个大图. 在节点维度里, 节点和其特征会 拼接 (concatenate) 在一起, 从而形成大图的节点信息:\n",
    "\n",
    "此程序与其他批处理程序相比有一些关键的优点:\n",
    "\n",
    "1.依赖于消息传递架构的GNN操作器不需要额外的修改. 这是因为不同图的节点之间不会进行消息交换.\n",
    "\n",
    "2. 没有计算或内存开销, 因为邻接矩阵是 稀疏的 (sparse), 也就是它只会保存非零的值, 也就是有连接的边.\n",
    "\n",
    "通过 torch_geometric.data.DataLoader 类, PyG会自动地处理, 构建多个子图成一个批量的大图.\n",
    "\n",
    "这里我们使用64的 batch_size, 这样我们会有3个小批量, 也就是包含 2 * 64 + 22 = 150 张图. 另外, 每个Batch对象都配有一个 赋值向量 (assignment vector) batch, 这个向量映射每个节点到它对应的批量里的图"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2owRWKcuoALo"
   },
   "source": [
    "![Screen Shot 2020-08-27 at 13.11.53.png]()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0gZ-l0npPIca",
    "outputId": "65b6d684-0e67-42f3-bf60-c98ec1d7338a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Step 1:\n",
      "=======\n",
      "Number of graphs in the current batch: 64\n",
      "DataBatch(edge_index=[2, 2540], x=[1141, 7], edge_attr=[2540, 4], y=[64], batch=[1141], ptr=[65])\n",
      "\n",
      "Step 2:\n",
      "=======\n",
      "Number of graphs in the current batch: 64\n",
      "DataBatch(edge_index=[2, 2542], x=[1154, 7], edge_attr=[2542, 4], y=[64], batch=[1154], ptr=[65])\n",
      "\n",
      "Step 3:\n",
      "=======\n",
      "Number of graphs in the current batch: 22\n",
      "DataBatch(edge_index=[2, 912], x=[419, 7], edge_attr=[912, 4], y=[22], batch=[419], ptr=[23])\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from torch_geometric.loader import DataLoader\n",
    "\n",
    "train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)\n",
    "test_loader = DataLoader(test_dataset, batch_size=64, shuffle=False)\n",
    "\n",
    "for step, data in enumerate(train_loader):\n",
    "    print(f'Step {step + 1}:')\n",
    "    print('=======')\n",
    "    print(f'Number of graphs in the current batch: {data.num_graphs}')\n",
    "    print(data)\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在对188个图，做成了3个batch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0pq-VyzUwPpw"
   },
   "source": [
    "### 训练 GNN\n",
    "\n",
    "要训练一个GNN来进行图分类, 我们通常需要以下步骤:\n",
    "\n",
    "1.通过执行多轮消息传递来嵌入每个节点\n",
    "\n",
    "2.聚集 (aggregate) 节点嵌入 (node embedding) 到一个统一的 图嵌入 (graph embedding), 也就是读出层 (readout layer)\n",
    "\n",
    "3.基于图嵌入来训练最终的分类器\n",
    "\n",
    "文献中已有很多不同的 读出层, 但是最常用的只是简单地利用了节点嵌入的优势:\n",
    "\n",
    "\n",
    "PyG也同样提供这一读出层 torch_geometric.nn.global_mean_pool, 该函数的输入为小批量内所有节点的节点嵌入和 赋值向量 batch 来为批量内的每张图计算图嵌入 (形状为 [batch_size, hidden_channels]). 将GNN应用于图分类任务的最终架构如下所示:\n",
    "\n",
    "$$\n",
    "\\mathbf{x}_{\\mathcal{G}} = \\frac{1}{|\\mathcal{V}|} \\sum_{v \\in \\mathcal{V}} \\mathcal{x}^{(L)}_v\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CN3sRVuaQ88l",
    "outputId": "59c998ba-b387-4413-d7d3-2101a27495bd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GCN(\n",
      "  (conv1): GCNConv(7, 64)\n",
      "  (conv2): GCNConv(64, 64)\n",
      "  (conv3): GCNConv(64, 64)\n",
      "  (lin): Linear(in_features=64, out_features=2, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "from torch.nn import Linear\n",
    "import torch.nn.functional as F\n",
    "from torch_geometric.nn import GCNConv\n",
    "from torch_geometric.nn import global_mean_pool\n",
    "\n",
    "\n",
    "class GCN(torch.nn.Module):\n",
    "    def __init__(self, hidden_channels):\n",
    "        super(GCN, self).__init__()\n",
    "        torch.manual_seed(12345)\n",
    "        self.conv1 = GCNConv(dataset.num_node_features, hidden_channels)\n",
    "        self.conv2 = GCNConv(hidden_channels, hidden_channels)\n",
    "        self.conv3 = GCNConv(hidden_channels, hidden_channels)\n",
    "        self.lin = Linear(hidden_channels, dataset.num_classes)\n",
    "\n",
    "    def forward(self, x, edge_index, batch):\n",
    "        # 1.对各节点进行编码\n",
    "        x = self.conv1(x, edge_index)\n",
    "        x = x.relu()\n",
    "        x = self.conv2(x, edge_index)\n",
    "        x = x.relu()\n",
    "        x = self.conv3(x, edge_index)\n",
    "\n",
    "        # 2. 平均操作\n",
    "        x = global_mean_pool(x, batch)  # [batch_size, hidden_channels]\n",
    "\n",
    "        # 3. 输出\n",
    "        x = F.dropout(x, p=0.5, training=self.training)\n",
    "        x = self.lin(x)\n",
    "        \n",
    "        return x\n",
    "\n",
    "model = GCN(hidden_channels=64)\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 300
    },
    "id": "HvhgQoO8Svw4",
    "outputId": "4d17af35-85c8-4bcd-8e16-c514fd2ff714"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 001, Train Acc: 0.6467, Test Acc: 0.7368\n",
      "Epoch: 002, Train Acc: 0.6467, Test Acc: 0.7368\n",
      "Epoch: 003, Train Acc: 0.6467, Test Acc: 0.7368\n",
      "Epoch: 004, Train Acc: 0.6467, Test Acc: 0.7368\n",
      "Epoch: 005, Train Acc: 0.6467, Test Acc: 0.7368\n",
      "Epoch: 006, Train Acc: 0.6533, Test Acc: 0.7368\n",
      "Epoch: 007, Train Acc: 0.7467, Test Acc: 0.7632\n",
      "Epoch: 008, Train Acc: 0.7267, Test Acc: 0.7632\n",
      "Epoch: 009, Train Acc: 0.7200, Test Acc: 0.7632\n",
      "Epoch: 010, Train Acc: 0.7133, Test Acc: 0.7895\n",
      "Epoch: 011, Train Acc: 0.7200, Test Acc: 0.7632\n",
      "Epoch: 012, Train Acc: 0.7200, Test Acc: 0.7895\n",
      "Epoch: 013, Train Acc: 0.7200, Test Acc: 0.7895\n",
      "Epoch: 014, Train Acc: 0.7133, Test Acc: 0.8421\n",
      "Epoch: 015, Train Acc: 0.7133, Test Acc: 0.8421\n",
      "Epoch: 016, Train Acc: 0.7533, Test Acc: 0.7368\n",
      "Epoch: 017, Train Acc: 0.7400, Test Acc: 0.7632\n",
      "Epoch: 018, Train Acc: 0.7133, Test Acc: 0.8421\n",
      "Epoch: 019, Train Acc: 0.7400, Test Acc: 0.7895\n",
      "Epoch: 020, Train Acc: 0.7533, Test Acc: 0.7368\n",
      "Epoch: 021, Train Acc: 0.7467, Test Acc: 0.7895\n",
      "Epoch: 022, Train Acc: 0.7467, Test Acc: 0.7895\n",
      "Epoch: 023, Train Acc: 0.7533, Test Acc: 0.7895\n",
      "Epoch: 024, Train Acc: 0.7267, Test Acc: 0.8421\n",
      "Epoch: 025, Train Acc: 0.7533, Test Acc: 0.7632\n",
      "Epoch: 026, Train Acc: 0.7533, Test Acc: 0.7632\n",
      "Epoch: 027, Train Acc: 0.7600, Test Acc: 0.8158\n",
      "Epoch: 028, Train Acc: 0.7533, Test Acc: 0.8421\n",
      "Epoch: 029, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 030, Train Acc: 0.7600, Test Acc: 0.8158\n",
      "Epoch: 031, Train Acc: 0.7600, Test Acc: 0.8158\n",
      "Epoch: 032, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 033, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 034, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 035, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 036, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 037, Train Acc: 0.7400, Test Acc: 0.7632\n",
      "Epoch: 038, Train Acc: 0.7667, Test Acc: 0.8158\n",
      "Epoch: 039, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 040, Train Acc: 0.7533, Test Acc: 0.7368\n",
      "Epoch: 041, Train Acc: 0.7467, Test Acc: 0.7368\n",
      "Epoch: 042, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 043, Train Acc: 0.7667, Test Acc: 0.8158\n",
      "Epoch: 044, Train Acc: 0.7533, Test Acc: 0.7632\n",
      "Epoch: 045, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 046, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 047, Train Acc: 0.7667, Test Acc: 0.8158\n",
      "Epoch: 048, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 049, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 050, Train Acc: 0.7667, Test Acc: 0.8158\n",
      "Epoch: 051, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 052, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 053, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 054, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 055, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 056, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 057, Train Acc: 0.7533, Test Acc: 0.7632\n",
      "Epoch: 058, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 059, Train Acc: 0.7800, Test Acc: 0.7632\n",
      "Epoch: 060, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 061, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 062, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 063, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 064, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 065, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 066, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 067, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 068, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 069, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 070, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 071, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 072, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 073, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 074, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 075, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 076, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 077, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 078, Train Acc: 0.7733, Test Acc: 0.8421\n",
      "Epoch: 079, Train Acc: 0.7667, Test Acc: 0.8158\n",
      "Epoch: 080, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 081, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 082, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 083, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 084, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 085, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 086, Train Acc: 0.7800, Test Acc: 0.8158\n",
      "Epoch: 087, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 088, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 089, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 090, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 091, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 092, Train Acc: 0.7800, Test Acc: 0.8158\n",
      "Epoch: 093, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 094, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 095, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 096, Train Acc: 0.7600, Test Acc: 0.7895\n",
      "Epoch: 097, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 098, Train Acc: 0.7733, Test Acc: 0.8158\n",
      "Epoch: 099, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 100, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 101, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 102, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 103, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 104, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 105, Train Acc: 0.7733, Test Acc: 0.7368\n",
      "Epoch: 106, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 107, Train Acc: 0.7733, Test Acc: 0.7105\n",
      "Epoch: 108, Train Acc: 0.8000, Test Acc: 0.7632\n",
      "Epoch: 109, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 110, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 111, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 112, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 113, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 114, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 115, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 116, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 117, Train Acc: 0.7733, Test Acc: 0.7895\n",
      "Epoch: 118, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 119, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 120, Train Acc: 0.8000, Test Acc: 0.7105\n",
      "Epoch: 121, Train Acc: 0.7600, Test Acc: 0.7632\n",
      "Epoch: 122, Train Acc: 0.7667, Test Acc: 0.7105\n",
      "Epoch: 123, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 124, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 125, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 126, Train Acc: 0.7733, Test Acc: 0.7368\n",
      "Epoch: 127, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 128, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 129, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 130, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 131, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 132, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 133, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 134, Train Acc: 0.7667, Test Acc: 0.7632\n",
      "Epoch: 135, Train Acc: 0.8067, Test Acc: 0.7368\n",
      "Epoch: 136, Train Acc: 0.7800, Test Acc: 0.7632\n",
      "Epoch: 137, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 138, Train Acc: 0.8133, Test Acc: 0.7105\n",
      "Epoch: 139, Train Acc: 0.7867, Test Acc: 0.7632\n",
      "Epoch: 140, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 141, Train Acc: 0.8000, Test Acc: 0.6579\n",
      "Epoch: 142, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 143, Train Acc: 0.7933, Test Acc: 0.7632\n",
      "Epoch: 144, Train Acc: 0.7867, Test Acc: 0.7368\n",
      "Epoch: 145, Train Acc: 0.8267, Test Acc: 0.7368\n",
      "Epoch: 146, Train Acc: 0.7667, Test Acc: 0.7895\n",
      "Epoch: 147, Train Acc: 0.7800, Test Acc: 0.7105\n",
      "Epoch: 148, Train Acc: 0.7933, Test Acc: 0.7895\n",
      "Epoch: 149, Train Acc: 0.8200, Test Acc: 0.7105\n",
      "Epoch: 150, Train Acc: 0.7800, Test Acc: 0.7895\n",
      "Epoch: 151, Train Acc: 0.7800, Test Acc: 0.7632\n",
      "Epoch: 152, Train Acc: 0.7867, Test Acc: 0.7632\n",
      "Epoch: 153, Train Acc: 0.8067, Test Acc: 0.7368\n",
      "Epoch: 154, Train Acc: 0.8067, Test Acc: 0.7368\n",
      "Epoch: 155, Train Acc: 0.7867, Test Acc: 0.7632\n",
      "Epoch: 156, Train Acc: 0.7800, Test Acc: 0.7105\n",
      "Epoch: 157, Train Acc: 0.8000, Test Acc: 0.7368\n",
      "Epoch: 158, Train Acc: 0.7800, Test Acc: 0.7368\n",
      "Epoch: 159, Train Acc: 0.7867, Test Acc: 0.7632\n",
      "Epoch: 160, Train Acc: 0.7867, Test Acc: 0.7632\n",
      "Epoch: 161, Train Acc: 0.7867, Test Acc: 0.7632\n",
      "Epoch: 162, Train Acc: 0.7933, Test Acc: 0.7632\n",
      "Epoch: 163, Train Acc: 0.7933, Test Acc: 0.7632\n",
      "Epoch: 164, Train Acc: 0.7867, Test Acc: 0.8158\n",
      "Epoch: 165, Train Acc: 0.7800, Test Acc: 0.8158\n",
      "Epoch: 166, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 167, Train Acc: 0.7733, Test Acc: 0.7632\n",
      "Epoch: 168, Train Acc: 0.7800, Test Acc: 0.7632\n",
      "Epoch: 169, Train Acc: 0.8000, Test Acc: 0.7632\n",
      "Epoch: 170, Train Acc: 0.7933, Test Acc: 0.7632\n"
     ]
    }
   ],
   "source": [
    "model = GCN(hidden_channels=64)\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.01)\n",
    "criterion = torch.nn.CrossEntropyLoss()\n",
    "\n",
    "def train():\n",
    "    model.train()\n",
    "\n",
    "    for data in train_loader:  # Iterate in batches over the training dataset.\n",
    "        out = model(data.x, data.edge_index, data.batch)  # Perform a single forward pass.\n",
    "        loss = criterion(out, data.y)  # Compute the loss.\n",
    "        loss.backward()  # Derive gradients.\n",
    "        optimizer.step()  # Update parameters based on gradients.\n",
    "        optimizer.zero_grad()  # Clear gradients.\n",
    "\n",
    "def test(loader):\n",
    "    model.eval()\n",
    "\n",
    "    correct = 0\n",
    "    for data in loader:  # Iterate in batches over the training/test dataset.\n",
    "        out = model(data.x, data.edge_index, data.batch)  \n",
    "        pred = out.argmax(dim=1)  # Use the class with highest probability.\n",
    "        correct += int((pred == data.y).sum())  # Check against ground-truth labels.\n",
    "    return correct / len(loader.dataset)  # Derive ratio of correct predictions.\n",
    "\n",
    "\n",
    "for epoch in range(1, 171):\n",
    "    train()\n",
    "    train_acc = test(train_loader)\n",
    "    test_acc = test(test_loader)\n",
    "    print(f'Epoch: {epoch:03d}, Train Acc: {train_acc:.4f}, Test Acc: {test_acc:.4f}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "3. Graph Classification.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
